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Program Description 1 



Cooperative Integrated Reading and Composition ® is a reading 
and writing program for students in grades 2 through 6. It has 
three principal elements: story-related activities, direct instruc- 
tion in reading comprehension, and integrated language arts/ 
writing. Daily lessons provide students with an opportunity to 
practice comprehension and reading skills in pairs and small 
groups. Pairs of students read to each other; predict how stories 



will end; summarize stories; write responses to questions posed 
by the teacher; and practice spelling, decoding, and vocabulary. 
Within cooperative teams of four, students work to understand 
the main idea of a story and work through the writing activities 
linked to the story. A Spanish version of the program is available 
for grades 2 through 5. 



Research 2 



Two studies of Cooperative Integrated Reading and Composi- 
tion® that fall within the scope of the Adolescent Literacy review 
protocol meet What Works Clearinghouse (WWC) evidence stan- 
dards with reservations. The two studies included approximately 
1,460 students in grades 2 through 6 who attended nine schools 3 
located in two school districts in the United States. 4 

Based on these two studies, the WWC considers the extent of 
evidence for Cooperative Integrated Reading and Composition® 



on adolescent learners to be medium to large for the compre- 
hension and general literacy achievement domains. No studies 
that meet WWC evidence standards with or without reservations 
examined the effectiveness of Cooperative Integrated Reading 
and Composition® on adolescent learners in the alphabetics or 
reading fluency domains. 



1. The descriptive information for this program was obtained from a publicly available source: the WWC Beginning Reading intervention report (July 2007). 
The WWC requests developers to review the program description sections for accuracy from their perspective. The program description was updated 
by the developer in December 2008. Further verification of the accuracy of the descriptive information for this program is beyond the scope of this 
review. The literature search reflects documents publicly available by March 2010. 

2. The studies in this report were reviewed using WWC Evidence Standards, Version 2.0 (see the WWC Procedures and Standards Handbook, Chapter III), 
as described in protocol Version 2.0. 

3. The Adolescent Literacy topic area reviews studies of interventions administered to students in grades 4-12 (or 9-18 years of age). For studies that 
include samples of students that span both the Adolescent Literacy (grades 4-12) and Beginning Reading (grades K-3) topic areas and cannot be disag- 
gregated by grade level, the Adolescent Literacy topic area also reviews any studies that include 5th-grade students or higher. For example, this report 
includes a combined sample of students from grades 2-6 (Jewell, 1994; Stevens & Slavin, 1995). 

4. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 
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Effectiveness Cooperative Integrated Reading and Composition® was found to have potentially positive effects on comprehension and general 
literacy achievement for adolescent learners. 



Alphabetics 


Reading fluency 


Comprehension 


General literacy 
achievement 



Rating of effectiveness 


na 


na 


Potentially positive 
effects 


Potentially positive 
effects 


Improvement index 5 


na 


na 


Average: +7 percentile 
points 


Average: +2 percentile 
points 




na 


na 


Range: +3 to +11 
percentile points 


Range: -3 to +8 
percentile points 

na = not applicable 



Additional program 
information 



Developer and contact 

Cooperative Integrated Reading and Composition® was 
developed in 1983 by Robert Slavin and Nancy Madden at the 
Center for Social Organization of Schools at The Johns Hopkins 
University. Cooperative Integrated Reading and Composition ® 
is distributed by the Success for All Foundation, Inc. Address: 
200 W. Towsontown Blvd., Baltimore, MD 21204-5200. Email: 
sfainfo@successforall.org. Web site: http://www.successforall. 
net/Programs/readingwings.html. Telephone: (800) 548-4998 
ext. 2372. 

Scope of use 

Cooperative Integrated Reading and Composition® was 
first used as part of a cooperative elementary whole-school 
reform model. The program was later reformulated as Reading 
Roots (for beginning readers) and Reading Wings (for upper 
elementary students) and is a component of both the Success 
for All comprehensive school reform model and a stand-alone 
reading program. 



Teaching 

The program uses daily 90-minute lessons to focus on story- 
related activities, direct instruction in reading comprehension, 
and integrated reading and language arts activities. In a team 
setting, mixed-ability students work together to read, discuss 
their reading to clarify unknown vocabulary, reread for fluency, 
understand the main idea, comprehend stories, and work through 
the writing process linked to the texts that the students are read- 
ing (including drafting, revising, and editing each other’s writing). 
Students are rewarded on the basis of the team’s performance to 
provide motivation to work together and help each other. 

Teacher training includes a two-day session that covers word 
structure and phonics, vocabulary development, fluency, and 
comprehension skills, as well as program management and 
cooperative learning strategies. Technical support by phone or 
on-site visits is also provided. 

Cost 6 

The cost of the program is approximately $150 per student for 
training and materials, depending on school size and the number 
of schools within a district that are participating. 



5. These numbers show the average and range of student-level improvement indices for all findings across the studies. 

6. The prices reported here are from the July 2007 Beginning Reading intervention report. 
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Research Fifty-two studies reviewed by the WWC investigated the effects 
of Cooperative Integrated Reading and Composition® on 
adolescent learners. Two studies (Jewell, 1994; Stevens & Slavin, 
1995) are quasi-experimental designs that meet WWC evidence 
standards with reservations. The remaining 50 studies do not 
meet either WWC evidence standards or eligibility screens. 

Meets evidence standards with reservations 
The studies included in this report evaluated Cooperative 
Integrated Reading and Composition® either as part of a coop- 
erative elementary whole-school reform model (Stevens & Slavin, 
1995) or as a stand-alone reading program (Jewell, 1994). 

Stevens and Slavin (1995) conducted a quasi-experiment that 
examined the effects of the Cooperative Integrated Reading 
and Composition® curriculum on students in grades 2 through 
6 attending five elementary schools in Maryland. Investigators 
matched two treatment schools with three comparison schools 
based on academic achievement, ethnicity, and socioeconomic 
background. Treatment schools implemented the Cooperative 
Elementary School model, a whole-school reform model that 
uses cooperative learning strategies across multiple content 
areas, in which Cooperative Integrated Reading and Com- 
position® was taught as a language arts/reading curriculum. 
Comparison schools implemented some components of the 
Cooperative Elementary School model but not the Cooperative 
Integrated Reading and Composition® curriculum. The WWC 
based its effectiveness ratings on findings from comparisons 
of 411 students in two schools who received Cooperative Inte- 
grated Reading and Composition ® and 462 comparison group 
students in three schools who received regular instruction. The 



study reported students’ outcomes after one and two years of 
program implementation. 7 

Jewell (1994) conducted a quasi-experiment that examined 
the effects of Cooperative Integrated Reading and Composition® 
on students in grades 2 through 6 attending four elementary 
schools in one school district in the United States. Teachers vol- 
unteered to participate in the study; however, a number of these 
teachers were placed in the comparison group because they 
elected not to implement Cooperative Integrated Reading and 
Composition®. The analytical sample included 15 classrooms 
whose students received Cooperative Integrated Reading and 
Composition® and 15 comparison classrooms whose students 
received the regular district-adopted reading and language arts 
program. The study reported students’ outcomes after seven to 
eight months of program implementation. 

Extent of evidence 

The WWC categorizes the extent of evidence in each domain 
as small or medium to large (see the WWC Procedures and 
Standards Handbook, Appendix G). The extent of evidence 
takes into account the number of studies and the total sample 
size across the studies that meet WWC evidence standards with 
or without reservations. 8 

The WWC considers the extent of evidence for Cooperative 
Integrated Reading and Composition® to be medium to large for 
comprehension and general literacy achievement for adolescent 
learners. No studies that meet WWC evidence standards with or 
without reservations examined the effectiveness of Cooperative 
Integrated Reading and Composition® in the alphabetics or 
reading fluency domains for adolescent learners. 



7. Two-year findings are considered for the effectiveness rating because these findings reflect the maximum exposure to the program. One-year findings 
are not included in this rating but are reported in Appendices A4.1 and A4.2. 

8. The extent of evidence categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the 
number and size of studies. Additional factors associated with a related concept— external validity, such as the students’ demographics and the types 
of settings in which studies took place— are not taken into account for the categorization. Information about how the extent of evidence rating was 
determined for Cooperative Integrated Reading and Composition® is in Appendix A6. 
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Effectiveness 



Findings 

The WWC review of interventions for Adolescent Literacy 
addresses student outcomes in four domains: alphabetics, 
reading fluency, comprehension, and general literacy achieve- 
ment. The studies included in this report cover two domains: 
comprehension and general literacy achievement. The findings 
below present the authors’ estimates and WWC-calculated 
estimates of the size and the statistical significance of the 
effects of Cooperative Integrated Reading and Composition® 
on adolescent learners. 9 

Comprehension. Stevens and Slavin (1995) reported, and 
the WWC confirmed, statistically significant positive effects 
of Cooperative Integrated Reading and Composition® on the 
Reading Comprehension and Vocabulary subtests of the Cali- 
fornia Achievement Test (CAT) for students in grades 2 through 
6. Jewell (1994) did not find statistically significant effects of 
Cooperative Integrated Reading and Composition® on the 
Reading Comprehension and Vocabulary subtests of the Gates- 
MacGinitie Reading Test for students in grades 2 through 6, and 
the WWC-calculated average effect across the two measures 
was not statistically significant or large enough to be considered 
substantively important according to WWC criteria (i.e., an effect 
size of at least 0.25). 10 Thus, for the comprehension domain, one 
study showed statistically significant positive effects, and one 
study showed indeterminate effects. 



General literacy achievement. Stevens and Slavin (1995) 
reported, and the WWC confirmed, a statistically significant pos- 
itive effect of Cooperative Integrated Reading and Composition® 
on the Language Expression subtest of the CAT for students in 
grades 2 through 6 but did not find statistically significant effects 
on the CAT Language Mechanics subtest. Jewell (1994) did not 
find statistically significant effects of Cooperative Integrated 
Reading and Composition® on the Reading Proficiency subtest 
of the Bass Academic Skills Sample for students in grades 2 
through 6. According to WWC calculations, the effect was not 
large enough to be considered substantively important accord- 
ing to WWC criteria. Thus, for the general literacy achievement 
domain, one study showed statistically significant positive 
effects, and one study showed indeterminate effects. 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as positive, potentially positive, mixed, no discernible 
effects, potentially negative, or negative. The rating of effective- 
ness takes into account four factors: the quality of the research 
design, the statistical significance of the findings, the size of 
the difference between participants in the intervention and the 
comparison conditions, and the consistency in findings across 
studies (see the WWC Procedures and Standards Handbook, 
Appendix E). 



9. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within 
classrooms or schools and for multiple comparisons. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and 
Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of 
Stevens and Slavin (1995), a correction for multiple comparisons was needed, so the significance levels may differ from those reported in the original 
study. As the authors used hierarchical linear modeling (HLM) analyses, which accounted for multi-level data (of students nested within classrooms and 
schools), correction for clustering was not needed. In the case of Jewell (1994), no corrections for clustering or multiple comparisons were needed. 

10. The WWC computes an average effect size as a simple average of the effect sizes across all individual findings within the study domain. 
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The WWC found Cooperative 
Integrated Reading and 
Composition® to have 
potentially positive effects 
on comprehension and 
general literacy achievement 
for adolescent learners 



Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and an 
average improvement index across studies (see WWC Proce- 
dures and Standards Handbook, Appendix F). The improvement 
index represents the difference between the percentile rank 
of the average student in the intervention condition and the 
percentile rank of the average student in the comparison condi- 
tion. Unlike the rating of effectiveness, the improvement index is 
entirely based on the size of the effect, regardless of the statisti- 
cal significance of the effect, the study design, or the analysis. 
The improvement index can take on values between -50 and 
+50, with positive numbers denoting favorable results for the 
intervention group. 



The average improvement index for comprehension is +7 per- 
centile points, with a range of +3 to +11 percentile points across 
findings from two studies. The average improvement index for 
general literacy achievement is +2 percentile points, with a range 
of -3 to +8 percentile points across findings from two studies. 

Summary 

The WWC reviewed 52 studies of Cooperative Integrated Read- 
ing and Composition ® for adolescent learners. Two studies meet 
WWC evidence standards with reservations; the remaining 50 
studies do not meet either WWC evidence standards or eligibility 
screens. Based on these studies, the WWC found potentially 
positive effects on comprehension and general literacy achieve- 
ment for adolescent learners. The conclusions presented in this 
report may change as new research emerges. 
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Appendix 



Appendix A1.1 Study characteristics: Stevens & Slavin, 1995 



Characteristic 


Description 


Study citation 


Stevens, R. J., & Slavin, R. E. (1995). The Cooperative Elementary School: Effects on students’ achievement, attitudes and social relations. American Educational 
Research Journal, 32(2), 321-351. 


Participants 


This study is a quasi-experiment conducted in five schools. Two treatment schools were selected by the investigators to implement the intervention, and three schools, 
matched on academic achievement, ethnicity, and socioeconomic background, were selected to serve as comparison schools. Classes in grades 2 through 6 in the treatment 
schools were matched with classes in the comparison schools based on pretest scores on the California Achievement Test (Reading, Language Arts, and Mathematics). The 
study’s analytic sample included 411 students in 21 treatment classrooms and 462 students in 24 comparison classrooms. The study reported students’ outcomes after two 
years of program implementation; these findings were used in the intervention ratings and can be found in Appendices A3.1 and A3. 2. Additional findings reflecting students’ 
outcomes after one year of program implementation can be found in Appendices A4.1 and A4.2 


Setting 


The study was conducted in five schools in one suburban school district in Maryland. The student populations of each school ranged from 4% to 15% minority students, and 
from 2% to 20% students received free or reduced-price lunch. 


Intervention 


Intervention schools implemented the Cooperative Elementary School model, a whole-school reform model that uses cooperative learning strategies across multiple 
content areas. Teachers used peer coaching and conducted their planning in a cooperative manner. Cooperative Elementary School emphasizes teacher involvement in 
site-based management and parent involvement in schools. The language arts/reading curriculum within Cooperative Elementary School is Cooperative Integrated Reading 
and Composition ®. Daily lessons, which focus on story-related activities, direct instruction in reading comprehension, and integrated reading and language arts activities, 
incorporate team practice, peer assessment, and team/partner recognition. This program was phased in gradually during the first year of the two-year implementation. 


Comparison 


Reading activities consisted of students working in small reading groups using a basal series, workbooks, worksheets, and activities based on teacher-prepared materials. 
Language arts activities generally involved whole-class instruction using a published language arts series, as well as teacher-developed activities. Comparison schools did 
not use structured cooperative learning during classroom instruction, although occasional cooperative activities were used by some of the teachers. Comparison schools 
implemented some of the components of the Cooperative Elementary School model, but they did not implement Cooperative Integrated Reading and Composition ®. 


Primary outcomes 
and measurement 


For both the pretest and the posttest, students took the Vocabulary, Reading Comprehension, Language Expression, and Language Mechanics subtests of the California 
Achievement Test. Scores were converted to z-scores in order to conduct analyses across the grades included in the study sample (grades 2-6). For a more detailed descrip- 
tion of these outcome measures, see Appendices A2.1 and A2.2. 


Staff/teacher training 


Intervention teachers were trained in Cooperative Integrated Reading and Composition ® prior to implementation. Subsequent trainings were conducted at two-month intervals 
during the school year. Trainers reviewed with treatment teachers a detailed manual that explains how to implement the program in the classroom. Trainers also provided 
simulated demonstrations of lessons. In addition, during the school year, members of the research staff observed treatment-group classes, participated in meetings with 
treatment-group teachers, and observed steering committee meetings in order to facilitate implementation of the program components. 
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Appendix A1.2 Study characteristics: Jewell, 1994 



Characteristic 


Description 


Study citation 


Jewell, M. E. (1994). The effect of classroom-based follow-up assistance on mainstream reading and language arts instruction (Doctoral dissertation, University of 
Washington, 1994). Dissertation Abstracts International, 55( 11A), 107-3473. 


Participants 


This study is a quasi-experiment that initially included a sample of 51 second- to sixth-grade classrooms assigned to one of three conditions: (1) comparison; (2) treatment, 
receiving Cooperative Integrated Reading and Composition ® training in the summer preceding implementation; and (3) treatment, receiving the program training as well as 
follow-up support during the school year. 1 The treatment classrooms were matched with comparison classrooms on the Gates— MacGinitie pretest scores. This review focuses 
on comparisons of the 15 classrooms taught by teachers who received either program training or program training with follow-up support, and the 15 classrooms in the 
comparison group. 


Setting 


The study took place in four schools in one district in the United States. The participating elementary schools served 9% to 27% minority students, and less than 15% of the 
student population received special education services. 


Intervention 


There were two forms of the Cooperative Integrated Reading and Composition ® intervention: (1) training only and (2) training plus follow-up support. The intervention 
group participated in teacher-led basal-related activities, partner reading, story-related writing, reading of words aloud, word meaning activities, story retelling, spelling, 
direct instruction in reading comprehension, home reading, integrated language arts and writing, weekly tests, and cooperative learning groups of four students. The 
program was implemented in intervention classrooms for seven to eight months. 


Comparison 


Comparison group teachers continued to teach in accordance with their own style and used the regular district-adopted reading and language arts program (basal materials). 
All comparison schools used the same reading and language arts curricula: Houghton Mifflin Reading ( Durr et at, 1989) 2 and the Silver Burdett & Ginn English senes (Ragno, 
Toth, & Gray, 1988). 3 


Primary outcomes 
and measurement 


For both the pretest and the posttest, students took the Gates— MacGinitie Reading Test Comprehension and Vocabulary subtests and the Basic Academic Skills Sample Read- 
ing Proficiency subtest. For a more detailed description of these outcome measures, see Appendices A2.1 and A2.2. 


Staff/teacher training 


The study’s investigator provided Cooperative Integrated Reading and Composition ® training and follow-up assistance. The training took place during five 4-hour sessions 
(spanning one week) during the summer preceding the study. Teachers were provided with lesson plans that were aligned to the district's basal series curriculum. Teachers 
practiced the Cooperative Integrated Reading and Composition ® components and received feedback from the investigator and peers. All Cooperative Integrated Reading and 
Compos/f/ori®-trained teachers participated in two follow-up meetings during the school year. Teachers assigned to receive classroom-based follow-up assistance also had 
the investigator observe lessons (on average, about 10 observations), provide feedback, demonstrate teaching procedures, and make recommendations for future lessons. 



1. Results from these analyses are not included in this report because the treatment groups and the comparison group were not equivalent at baseline. 

2. Durr, W. K., Pikulski, J. J., Bean, R. M., Cooper, J. D., Glaser, N. A., Greenlaw, M. J., & Schoephoerster, H. (1989). Houghton Mifflin Reading. Boston: Houghton Mifflin, 

3. Ragno, N. N., Toth, M. D., & Gray, B. G. (1988). Silver Burdett & Ginn English. Morristown, NJ: Silver Burdett & Ginn. 



WWC Intervention Report Cooperative Integrated Reading and Composition® August 2010 





Appendix A2.1 Outcome measures for the comprehension domain 



Outcome measure Description 



Reading comprehension construct 

California Achievement The CAT is a norm- and criterion-referenced annual test. The Reading Comprehension subtest is administered to grades 1 through 12 and focuses on students' use of reading 



Test (CAT) — Reading 
Comprehension subtest 


comprehension strategies. Passages reflect a wide range of narrative, expository, contemporary, and traditional texts. The test measures information recall, meaning construc- 
tion, form analysis, and meaning evaluation of seven different selections (as cited in Stevens & Slavin, 1995). 


Gates-MacGinitie Reading 
Tests — Comprehension 
subtest 


The Comprehension subtest of the Gates-MacGinitie Reading Test measures each student’s ability to read and understand different types of prose. The test contains 11 
passages of various lengths and subjects and 48 questions (as cited in Jewell, 1994). 



Vocabulary development construct 

California Achievement Test The CAT is a norm- and criterion-referenced annual test. The Vocabulary subtest contains 20 items measuring same-meaning, opposite-meaning words; multi-meaning 



(CAT)— Vocabulary subtest 


words; words in context; and the meaning of affixes (as cited in Stevens & Slavin, 1995). 


Gates-MacGinitie Reading 
Tests— Vocabulary subtest 


The Vocabulary subtest of the Gates-MacGinitie Reading Test measures a student's reading vocabulary. The test contains 45 questions that measure word knowledge 
by asking students to choose one word or phrase that means most nearly the same as a presented word. The vocabulary test takes 20 minutes to administer (as cited in 
Jewell, 1994). 



Appendix A2.2 Outcome measures for the general literacy achievement domain 



Outcome measure 


Description 


California Achievement 
Test (CAT)— Language 
Mechanics subtest 


The Language Mechanics and Language Expression subtests of the CAT work together to measure a broad range of language and writing skills, including the ability to apply 
standard usage and writing conventions and to develop effective sentences and paragraphs. The Language Mechanics subtest contains 20 items that measure skills in the 
mechanics of capitalization and punctuation. Editing skills are measured in the context of passages presented in various formats (for grades 4-12) (as cited in Stevens & 
Slavin, 1995). 


California Achievement 
Test (CAT)— Language 
Expression subtest 


The Language Mechanics and Language Expression subtests of the CAT work together to measure a broad range of language and writing skills, including the ability to apply 
standard usage and writing conventions and to develop effective sentences and paragraphs. The Language Expression subtest contains 20 items that measure skills in 
language usage and sentence structure. The items measure skills in the use of various parts of speech, formation and organization of sentences and paragraphs, and writing 
for clarity (for grades 4-12) (as cited in Stevens & Slavin, 1995). 


Basic Academic Skills 
Sample — Reading 
Proficiency subtest 


The Basic Academic Skills Sample is a group-administered, curriculum-based assessment of students' reading proficiency, measured via maze procedure using three pas- 
sages averaging 263 words. In this maze procedure, every seventh word in the passage is replaced by a multiple choice question item containing the correct word and two 
distractors. Students’ scores reflect the number of correct multiple choice selections, given one minute per passage (as cited in Jewell, 1994). 
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Appendix A3.1 Summary of study findings included in the rating for the comprehension domain 1 









Authors’ findings from the study 














Mean outcome 
(standard deviation) 2 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(classrooms/ 
students) 


Cooperative 
Integrated 
Reading and 
Composition® 
group 


Comparison 

group 


Mean 

difference 3 
(Cooperative 
Integrated 
Reading and 
Composition® 
- comparison) 


Statistical 

Effect significance 5 

size 4 (at a = 0.05) 


Improvement 

index 6 









Stevens & Slavin, 1995 78 










CAT — Reading Vocabulary 


Grades 2-6 


45/873 


0.10 

(0.98) 


-0.11 

(1.01) 


0.21 


0.21 


Statistically 

significant 


+8 


CAT — Reading Comprehension 


Grades 2-6 


45/873 


0.15 

(0.98) 


-0.13 

(1.01) 


0.28 


0.28 


Statistically 

significant 


+11 


Average for comprehension (Stevens & Slavin, 1995)' 


9 








0.25 


Statistically 

significant 


+10 


GMRT — Vocabulary subtest 


Grades 2-6 


30 classrooms 


50.48 

(21.06) 


Jewell, 1994 7,10 

49.07 

(21.06) 


1.41 


0.07 


ns 


+3 


GMRT — Comprehension subtest 


Grades 2-6 


30 classrooms 


52.20 

(21.06) 


49.16 

(21.06) 


3.04 


0.14 


ns 


+6 


Average for comprehension (Jewell, 1994) 9 










0.11 


ns 


+4 


Domain average for comprehension across all studies 9 








0.18 


na 


+7 



ns = not statistically significant 
na = not applicable 
CAT = California Achievement Test 
GMRT = Gates-MacGinitie Reading Test 

1. This appendix reports findings considered for the effectiveness rating and the average improvement indices for the comprehension domain. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 

had more similar outcomes. 

3. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

4. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

5. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

6. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 

The improvement index can take on values between -50 and +50, with positive numbers denoting favorable results for the intervention group. 



(continued) 
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Appendix A3.1 Summary of study findings included in the rating for the comprehension domain (continued) 

7. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple 
comparisons. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and 
Standards Handbook, Appendix D for multiple comparisons. In the case of Stevens and Slavin (1995), correction for multiple comparisons was needed, so the significance levels may differ 
from those reported in the original study. A correction for clustering was not needed, as the authors used HLM analyses, which accounted for multi-level data (of students nested within class- 
rooms and schools). In the case of Jewell (1994), no corrections for clustering or multiple comparisons were needed, as analyses were performed at the classroom level and findings were not 
statistically significant. 

8. The intervention group and comparison group mean outcome values for Stevens and Slavin (1995) are the HLM-fitted two-year posttest means. 

9. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated 
from the average effect sizes. 

10. The group mean values reported for Jewell (1994) are the pretest group means plus the gain scores. Standard deviations of 21.06 are from the normative student sample. 
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